Framework for Accelerometer Based Gesture Recognition and Seamless Integration with Desktop Applications

Accelerometer is one of the prominent sensors which are commonly embedded in new age handheld devices. Accelerometer measures acceleration forces in three orthogonal axes X, Y, Z. The raw acceleration values obtained due to the movement of device in 3D space which is hosting accelerometer can be used to interact and control wide range of applications running on the device and can also be integrated with desktop applications to enable intuitive ways of interaction. The goal of the project is to build a generic and economic, gesture recognition framework based on accelerometer sensor and enable seamless integration with desktop applications by providing natural ways of interaction with desktop applications based on the gesture information obtained from accelerometer sensor embedded in Smartphone device held in user's hand. This framework provides an alternative to the conventional interface devices like mouse, keyboard and joystick. With the integration of gesture recognition framework with desktop applications user can remotely play games, create drawings, control key and mouse event based applications. And since this is a generic framework, it can be integrated with any of the existing desktop applications irrespective of whether the application exposes APIs or not, or whether it is a legacy or a newly programmed application. A communication protocol is required to transfer Accelerometer data from handheld device to desktop computer, and this can be achieved either through Wi-Fi or Bluetooth communication protocol. The project achieves data transmission between handheld device and desktop computer through "Bluetooth" protocol. Once the Accelerometer data is received at desktop computer, the raw data is initially filtered and processed into appropriate gesture information after many computations through multiple algorithms. The key event publisher will take the processed gestures as input and converts them into appropriate events and publishes them on to the target applications to be controlled. This framework makes interaction with desktop applications very natural and intuitive. And it also enables game and application developers to build creative games and applications which are highly engaging.